214 research outputs found
Learning to Reach Agreement in a Continuous Ultimatum Game
It is well-known that acting in an individually rational manner, according to
the principles of classical game theory, may lead to sub-optimal solutions in a
class of problems named social dilemmas. In contrast, humans generally do not
have much difficulty with social dilemmas, as they are able to balance personal
benefit and group benefit. As agents in multi-agent systems are regularly
confronted with social dilemmas, for instance in tasks such as resource
allocation, these agents may benefit from the inclusion of mechanisms thought
to facilitate human fairness. Although many of such mechanisms have already
been implemented in a multi-agent systems context, their application is usually
limited to rather abstract social dilemmas with a discrete set of available
strategies (usually two). Given that many real-world examples of social
dilemmas are actually continuous in nature, we extend this previous work to
more general dilemmas, in which agents operate in a continuous strategy space.
The social dilemma under study here is the well-known Ultimatum Game, in which
an optimal solution is achieved if agents agree on a common strategy. We
investigate whether a scale-free interaction network facilitates agents to
reach agreement, especially in the presence of fixed-strategy agents that
represent a desired (e.g. human) outcome. Moreover, we study the influence of
rewiring in the interaction network. The agents are equipped with
continuous-action learning automata and play a large number of random pairwise
games in order to establish a common strategy. From our experiments, we may
conclude that results obtained in discrete-strategy games can be generalized to
continuous-strategy games to a certain extent: a scale-free interaction network
structure allows agents to achieve agreement on a common strategy, and rewiring
in the interaction network greatly enhances the agents ability to reach
agreement. However, it also becomes clear that some alternative mechanisms,
such as reputation and volunteering, have many subtleties involved and do not
have convincing beneficial effects in the continuous case
Differentiable Game Mechanics
Deep learning is built on the foundational guarantee that gradient descent on
an objective function converges to local minima. Unfortunately, this guarantee
fails in settings, such as generative adversarial nets, that exhibit multiple
interacting losses. The behavior of gradient-based methods in games is not well
understood -- and is becoming increasingly important as adversarial and
multi-objective architectures proliferate. In this paper, we develop new tools
to understand and control the dynamics in n-player differentiable games.
The key result is to decompose the game Jacobian into two components. The
first, symmetric component, is related to potential games, which reduce to
gradient descent on an implicit function. The second, antisymmetric component,
relates to Hamiltonian games, a new class of games that obey a conservation law
akin to conservation laws in classical mechanical systems. The decomposition
motivates Symplectic Gradient Adjustment (SGA), a new algorithm for finding
stable fixed points in differentiable games. Basic experiments show SGA is
competitive with recently proposed algorithms for finding stable fixed points
in GANs -- while at the same time being applicable to, and having guarantees
in, much more general cases.Comment: JMLR 2019, journal version of arXiv:1802.0564
- …